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ABSTRACT 

This  paper  discusses  initial  efforts  to  understand  and  exploit 
Cognitive  Neuroscience  (CNS)  for  underwater  systems. 
Compared  to  early  sonars  that  used  electro-mechanical 
compensator  plates,  underwater  systems  have  benefited  from 
automation  using  digital  beamforming  and  signal  processing 
introduced  by  modem  electronics.  Some  of  the  evolving  body 
of  knowledge  about  CNS  and  how  animal  (including  human) 
brains  function  has  been  converted  into  practical,  computer- 
based  demonstrations  showing  how  to  improve  sonar  signal 
processing.  Additionally,  we  anticipate  that  a  growing 
understanding  of  CNS  will  lead  to  even  greater  automation, 
perhaps  even  Fully  Automated  Systems  Technology  (FAST) 
concepts.  FAST,  in  turn,  will  likely  revolutionize  sonar. 

1.  INTRODUCTION 

We  report  here  on  efforts  to  understand  and  exploit  Cognitive 
Neuroscience  (CNS)  for  application  to  underwater  systems,  in 
general,  and  to  underwater  acoustics,  in  particular.  Some  of  the 
evolving  body  of  knowledge  about  CNS  and  how  animal 
brains  function  has  been  converted  into  practical,  computer- 
based  demonstrations  showing  how  to  improve  sonars.  An 
application  of  such  a  sonar  is  monitoring  fish  populations  for 
treaty  control.  We  anticipate  that  a  growing  understanding  of 
CNS  will  lead  to  even  greater  automation;  indeed,  we  feel  that 
a  “Grand  Challenge”  is  to  develop  Fully  Automated  Systems 
Technology  (FAST)  concepts  that  will  revolutionize  sonar 
signal  processing.  This  anticipation  is  driven  by  system  needs 
and  brain  research  that  will  be  discussed  in  this  article. 
Gazzaniga  [9]  expressed  widely-held  thoughts  with  these 
words:  "At  some  point  in  the  future,  Cognitive  Neuroscience 
(CNS)  will  be  able  to  describe  the  algorithms  that  drive 
structural  neural  elements  into  the  physiological  activity  that 


results  in  perception,  cognition  and  perhaps  even 
consciousness."  The  application  of  CNS  to  sonar  systems  is 
driven  by  a  need  to  reduce  the  load  on  the  sonar  operator,  to 
simplify  the  sonar's  operation,  indeed  to  even  dramatically 
reduce  the  number  of  sonar  operators  while  simultaneously 
maintaining  or  improving  the  sonar's  performance  in  an 
increasingly  complex  environment.  We  expect  to  apply 
algorithms  of  some  structural  artificial  neural  elements  such  as 
sensory,  motor,  attention,  memory,  thought,  and  imagery 
systems  into  elements  of  a  sonar  system. 

2.  BACKGROUND 

2.1  System  Requirements 

This  paper  is  a  summary  of  the  current  state  of  CNS  work  of 
scientists  and  engineers  engaged  in  “Systems  Research”. 
These  efforts  are  at  the  systems  level  and  involve  work  across 
scientific  disciplines.  We  summarize  Systems  Research  in  the 
initial  stages  of  a  multi-year  examination  of  the  implications  of 
CNS  on  the  design  of  sonars.  We  have  focused  on  sonar 
systems,  as  a  first  logical  step  and  intend  to  expand  to  other 
undersea  systems  including,  for  example.  Autonomous 
Underwater  Vehicles  (AUVs),  at  a  later  date.  We  hope  to 
understand  how  the  brain  processes  data  and  investigate 
opportunities  to  use  this  information  to  improve  sonars  and 
other  underwater  systems.  Several  requirements  will  change  the 
way  the  sonar  operator  interacts  with  underwater  acoustic 
sensor  systems.  These  requirements  include: 

•  reducing  the  number  of  personnel, 

•  providing  information,  rather  than  processed  data,  to 
significantly  reduce  data  evaluation  time  and  thereby 
enhance  the  decision-making  process, 

•  increasing  sonar  array  size  to  improve  detection  which 
will,  as  a  by-product,  increase  the  sonar  operator  load, 
especially  in  cluttered  acoustic  environments,  and 
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•  increasing  focus  on  reconnaissance  connectivity  which 
places  new  and  complex  demands  on  operators. 


Figure  1.  Diagram  of  a  CNS  sonar,  modified  from  [6].  Note 
the  use  of  feedback,  memory,  alertment,  and  a  world  model. 


These  requirements  will  force  a  sonar  operator  paradigm  shift. 
This  shift  can  be  realized  by  further  automation  including 
taking  advantage  of  highly  developed  computer  models  of  how 
the  brain  functions.  At  this  initial  stage  of  our  work,  we  seek 
to  extend  and  apply  these  models: 

(1)  Scene  Analysis  (Bregman  [3]  and  Wang  [16]), 

(2)  Dynamically  Stable  Associative  Learning  (DYSTAL) 
(Vogl  and  Blackwell  [15]), 

(3)  Adaptive  Resonance  Theory  (ART)  (  Grossberg  [10]), 

(4)  Microtubules  (Hameroff  [11]),  and 

(5)  Automatic  Salient  Contour  Extraction  NeTwork 
(ASCENT)  (Finkel  [7]). 

These  models  are  a  representative  selection  that  we  have  found 
interesting  and  useful.  We  expect  to  include  other  models  in 
our  consideration  as  we  become  aware  of  them. 

2.2  CNS  Defined 

Kosslyn  and  Koenig  [14]  give  a  detailed  definition  of  CNS.  A 
more  comprehensive  discussion  and  extensive  compendium  of 
papers  is  provided  by  Gazzaniga  [9]  in  The  Cognitive 
Neurosciences.  Briefly,  cognition  is  the  process  of  perceiving 
and  knowing.  Neuroscience  is  the  investigation  of  how  the 
brain  works  using  techniques  and  results  horn  many 
disciplines.  Thus,  CNS  is  the  combination  of  cognition  and 
neuroscience  in  the  investigation  of  how  the  brain  performs  the 
cognition  process,  including  how  it  perceives,  learns, 
remembers,  reasons,  and  satisfies  goals.  Perceiving  includes 
abstracting,  classifying,  generalizing,  and  synthesizing. 
Learning  can  be  supervised  or  self-taught  and  includes 
cognitive  learning.  Remembering  uses  very  short  term,  short 
term,  long  term,  and  associative  memory.  Reasoning  involves 
making  analogies,  using  common  sense  and  prediction. 
Satisfying  goals  requires  planning,  decision  making,  and  self- 
motivation. 

3.  Principles  Drawn  From  CNS  Related  to 
Sonar  Experimentation 

The  brain  analyzes  incoming  acoustic  data  from  the  ears  to 
extract  sets  of  primitives,  which  are  sensations  that  it  cannot 


break  into  more  primitive  sensations.  In  auditory  (visual)  scene 
analysis,  a  primitive  is  a  stream  (visual  object)  that  cannot  be 
broken  into  parts.  In  Cooke's  Ph.D.  thesis  [5],  the  primitives 
were  synchrony  strands.  Features,  in  contrast,  are  properties  of 
streams  or  objects  that  can  be  used  to  classify  and  group  them. 
Features  might  include  pitch,  onset  and  offset  For  example, 
Cooke  uses  pitch  to  group  synchrony  strands  as  coming  from 
one  acoustic  source.  The  brain  extracts  primitives  by  measuring 
certain  features  of  the  incoming  data  using  appropriate  neural 
networks.  Artificial  neural  networks  attempt  to  emulate  actual 
brain  neural  networks,  weight,  sum,  then  distort  with  a  no¬ 
memory  non-linearity.  For  a  more  detailed  description  see 
Chang,  Bosworth,  and  Carter  [4]. 

The  human  brain,  through  its  complex  evolution,  appears  to 
have  followed  some  essential  systematic  principles.  We 
hypothesize  the  following  principles  as  the  possible  reasoning 
structure:  Incoming  data  from  sensors,  including  audio  and 
visual  is  fragmented  into  essential  elements  called 
“primitives.”  Stored  memories  reside  as  elementary  fragments 
located  separately  over  regions  of  the  brain’s  cortex.  A 
memory  location  specific  to  a  fragment  is  common  to  all 
concepts  in  memory  that  relate  to  that  description.  A  neuronal 
process  exists  which  matches  and  compares  fragmentary 
components  of  data  and  memory.  When  sufficient  preferential 
elements  are  present,  the  neuron  fires  announcing  the  rudiments 
of  a  concept  for  higher  level  assessment.  A  hierarchical 
analysis  exists  linking  the  fragmentation  of  data  to  alertment  in 
the  conscious  mind.  Neurons  exhibit  synaptic  plasticity. 
Under  special  circumstances,  dendritic  growth  can  occur.  The 
brain  has  feedforward  and  feedback  mechanisms.  The  brain 
operates  in  a  conscious  (or  alerted)  and  subconscious  (or 
unalerted)  mode.  Trial  solutions  are  attempted.  When  faced 
with  a  complex  set  of  incoming  data,  the  brain  selects  the 
response  that  provides  a  match  consistent  with  all  constraints. 


4.  MODELS  OF  THE  BRAIN 

In  this  section  we  report  on  five  models  of  brain  and  sensor 
structure:  (1)  Scene  Analysis,  (2)  Dynamically  Stable 
Associative  Learning  (DYSTAL),  (3)  Adaptive  Resonance 
Theory  (ART),  (4)  Microtubules,  and  (5)  ASCENT. 

4.1  Scene  Analysis 

Scene  Analysis  includes  visual  and  auditory  components. 
Visual  scene  analysis  is  the  separation  of  incoming  light 
energy  into  a  set  of  independent  visual  objects.  Similarly, 
Auditory  Scene  Analysis  (ASA)  is  the  separation  of  acoustic 
energy  into  a  set  of  independent  sound  sources.  Both  have 
potential  in  the  design  of  improved  sonar  systems  that  use,  for 
example,  either  automated  ASA  of  beam-formed  sonar  signals  or 
automated  visual  scene  analysis  of  displayed  sonar  grams. 
Scene  analysis  detects  primitives  (sub-objects  that  cannot  be 
further  decomposed)  and  combines  them  to  form  visual  objects 
or  sound  sources.  While  visual  and  auditory  primitives  are 
derived  largely  from  neuroscience,  high-level  fusion  ot 
analysis  is  carried  out  using  techniques  from  cognitive  science. 
Neuroscience  explains  critical  low-level  visual  processing  in 
the  brain  and  explains  some  facts  about  low-level  auditory 
processing.  For  underwater  acoustic  applications,  much  low- 
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level  auditory  processing  may  be  inferred  from  lower  level 
visual  processing.  Recent  advances  in  auditory  processing 
include  the  Ph.D.  thesis  by  Ellis  [6]  on  ASA. 

Ellis  [6]  states  that  his  thesis  “presents  a  collection  of 
components  assembled  into  a  partial  model  that  mimics  some  of 
the  aspects  of  auditory  information  processing.”  His  thesis  is 
comprised  of  two  notions:  “perception  proceeds  via  ... 
reconciliation  of  the  internal  representation  with  the  perceived 
(sensed)  signal  from  the  external  world”  and  “the  full  spectrum 
of  “real”  sounds  are  adequately  spanned  by  the  combinations 
of  a  few  simple  parameterized  sound  primitives.”  The  Ellis 
thesis  is  then  “prediction-driven”  unlike  the  ASA  data-driven 
models  of  Cooke  [5]  and  others  that  grew  out  of  Bregman  [3]. 
Ellis  develops  a  model  of  the  mind  that  is  constantly  trying  to 
reconcile  what  it  hears  with  what  it  expects,  it  then  becomes 
alerted  by  unexpected  sounds. 

4.2  Dynamically  Stable  Associative  Learning  (DYSTAL) 

DYSTAL  is  a  class  of  biologically  plausible  artificial  neural 
networks  based  on  studies  of  the  nervous  system  of  the  marine 
snail  Hermissenda  crassicomis  and  the  hippocampus  of  rabbits. 
(See  Alkon  [2]  and  Irwin  [12],  For  other  hippocampus  work, 
see  also  Japkowicz,  Myers,  and  Gluck  [13].)  DYSTAL 
networks,  unlike  standard  artificial  neural  networks,  use  non- 
Hebbian  learning  rules,  are  self-organizing,  converge 
monotonically,  have  large  capacity  and  linear  complexity  and 
require  few  training  samples.  A  DYSTAL  network  leams  to 
associate  conditioned  stimulus  input  patterns  with 
unconditioned  stimulus  patterns,  demonstrating  the 
association  by  outputting  the  appropriate  unconditioned 
stimulus  pattern  in  response  to  each  conditioned  stimulus 
input  pattern  alone.  Both  conditioned  and  unconditioned 
stimuli  are  vectors.  A  DYSTAL  network  has  input  neurons  for 
each  component  of  these  vectors  and  an  output  neuron  for  each 
component  of  the  unconditioned  stimulus.  Connecting  each 
output  neuron  with  its  corresponding  input  unconditioned 
stimulus  neuron  and  all  conditioned  stimulus  neurons  is  a  set 
of  patches,  which  are  vectors  that  act  as  hidden  neurons.  Prior 
to  training,  no  patches  exist  During  training,  DYSTAL 
compares  conditioned-stimulus/unconditioned-stimulus  pairs 
with  existing  patches.  If  any  existing  patches  match  the  stimuli 
sufficiently,  the  most  similar  patch  for  each  output  neuron  will 
determine  its  output.  DYSTAL  updates  each  such  selected 
patch  by  averaging  its  stored  patch  vector  with  appropriate 
components  of  the  stimuli.  Otherwise,  DYSTAL  creates  a  new 
patch  for  each  output  neuron  using  the  appropriate  components 
of  the  stimuli. 

In  a  presentation  at  NUWC  on  DYSTAL,  Vogl  and  Blackwell 
described  how  they  have  reverse  engineered  the  brain  to 
achieve  pattern  recognition  comparable  to  that  of  human 
observers.  Details  of  the  research  were  beginning  to  emerge  in 
the  Biological  Cybernetics  pattern  recognition  publication  by 
Alkon,  Blackwell,  Barbour,  Rigler,  and  Vogl  [1].  One 
experiment  involved  using  the  picture  of  a  face.  In  the 
experiment  comparing  the  DYSTAL  algorithm  to  humans, 
humans  slightly  outperform  DYSTAL  under  very  low  noise 
conditions  but  DYSTAL  outperforms  humans  (in  the  important 
case)  when  high  noise  levels  are  present,  a  case  of  interest  in 
underwater  applications. 


43  Adaptive  Resonance  Theory  (ART) 

Grossberg,  the  father  of  ART,  provides  the  following  acoustic 
example  (See  Grossberg  [10].)  “Suppose  that  you  hear  a  noise 
followed  by  the  words  “eel  is  on  the  ...”  If  followed  by  the 
word  “orange,”  you  hear  “peel  is  on  the  orange.”  If  followed 
by  the  word  “wagon”,  then  you  hear  “wheel  is  on  the  wagon.” 
Thus  it  appears  the  brain  can  work  “backward  in  time.” 
Grossberg ’s  theory  is  that  the  brain’s  neural  network 
resonates  for  a  short  while  until  it  can  resolve  this  uncertainty. 
Freedman  (See  Freedman  [8])  explains:  "  ...  ART— which  stands 
for  'adaptive  resonance  theory'  —  ...  consists  of  four  main 
components:  a  short  term  memory  ...  a  long  term  memory  ...  an 
attentional  subsystem  to  focus  attention  ...  and  a  reorienting 
system  ...  unlike  other  neural  networks,  ART  doesn't  depend  on 
an  outside  supervisor  to  tell  it  if  its  guess  was  right  ...  ART 
generally  blows  away  the  competition:  in  applications  for 
which  back-prop  networks  require  20,000  supervised  training 
rounds  „  ART  teaches  itself  in  3  rounds  ...  rivaling  human 
performance  ...  "  Others  would  argue  that  this  is  an  overly 
optimistic  assessment  of  ART  and  that  its  success  is 
situationally  dependent;  moreover,  that  competitive  methods, 
including  artificial  neural  networks  with  back  propagation  and 
more  advanced  versions  of  ART  are  needed  to  begin  to  rival 
portions  of  human  brain  performance. 

4.4  Microtubules 

Microtubules  are  being  modeled  as  small  computers  within 
neurons.  Freedman  [8]  discusses  Microtubules;  he  states:  "  ... 
microtubules  -  thin  cylinders  of  protein  molecules  that  lend 
internal  structure  to  neurons  ...  each  neuron's  microtubule 
network  in  effect  comprises  a  powerful,  high-speed  internal 
computer."  Freedman  quotes  researcher  S.  Hameroff  as  stating: 
"Neurons  produce  a  yes-or-no  decision  but  a  lot  of  thinking 
goes  into  that  decision.  Microtubules  could  be  the  primary 
processing  element  in  the  brain.”  Freedman  goes  on  to  suggest 
that:  "The  existence  of  microtubular  intelligence  within 

neurons  would  be  exceedingly  bad  news  for  any  researchers 
who  hopre  to  construct  an  artificially  intelligent  device  out  of 
conventional  computer  hardware.”  Hameroff  believes  that 
"The  complexity  of  neurons  ...  are  closer  to  entire  computers 
than  individual  switches.  This  implies  the  mechanism  of 
consciousness  may  depend  on  an  understanding  of  the 
organization  of  adaptive  (‘cognitive’)  functions  within  living 
cells.”  Hameroff  calculates  that  at  the  current  rate  of  progress, 
semiconductor  technology  will  produce  a  compruter  capable  of 
human-brain-like  raw  processing  power  ..  within  the  next  few 
decades  ...  (but)  if  each  neuron  has  sophisticated  internal 
processing  network  based  on  Microtubules,  he  claims,  then  the 
semiconductor  computers  of  the  early  Twenty-First  century 
will  fall  short  by  a  factor  of  more  than  a  billion.  While  we 
continue  to  find  evidence  that  there  are  sub-neuron 
computations  that  are  necessary  and  at  work,  we  have  not  yet 
found  the  experimental  evidence  that  mircotubules  are  the 
computational  machines  hypothesized. 

43  ASCENT 

ASCENT  (Automatic  Salient  Contour  Extraction  NeTwork)  [7] 
is  a  biologically  plausible  neural  network  based  upon  the 
human  visual  processing  system  that  is  capable  of  extracting 
salient  contours  from  real  images. 
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Figure  2.  Modeled  after  the  human  visual  processing 
system,  ASCENT  extracts  salient  contours  from  an 
actual  BTR  display 

It  has  been  used  to  extract  tracks  from  a  Bearing-Time  Recorder 
(BTR)  display  as  is  shown  in  Fig.  2.  hi  modem  sonars 
operators  must  page  through  many  similar  displays  of  the 
acoustic  scene.  Often,  for  example,  these  are  frequency-time 
plots  of  acoustic  power  displayed  for  each  of  numerous 
bearings.  Automated  methods  such  as  ASCENT  offer  the 
potential  to  replace  human  operators  with  Fully  Automated 
Systems  Technology  (FAST)  operators. 

5.  CONCLUSIONS 

Advances  in  CNS  have  been  and  will  continue  to  be  important 
to  Fully  Automated  Systems  Technology  (FAST)  including 
sonar  signal  processing.  Indeed,  biologically  inspired 
architectures,  even  if  not  precise  models  of  the  actual  biology 
can  suggest  new  sonar  processing  approaches  to  automation. 
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Purpose 


Introduce  the  Audience  to: 

•  What  the  computational 
underwater  hearing,  that  is,  sonar 
community  has  learned  about 
Cognitive  Neuroscience  (CNS) 

•  What  research  needs  to  be  done 


FAST 

GROUP 
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NUWC’s  CNS  Technical  Leaders 

(CNS  =  Cognitive  Neuroscience) 


8  JUA  co-authors 
Dr.  Cliff  Carter,  Sub  Sonar 
John  Cooke,  Surface  Sonar 
Dr.  Ivars  Kirsteins,  Surface  Sonar 
Dr.  Lew  Meier,  Sub  Sonar 
John  Sanchis,  Sub  Sonar 
Dr.  James  Shores,  Sub  Sonar 
Dr.  Azizul  H.  Quazi,  Surface  Sonar 
Roger  Woodall,  Sub  Sonar 


Other  contributors 
Dr.  Arnie  Aaron,  Weapons 
Dr.  Y.  Masakowski,  Combat  Control 
Chris  DeAngeiis,  Combat  Control 
Dr.  Richard  Katz,  Surface  Sonar 
Dr.  Roy  Streit,  Combat  Control 
Dr.  Susan  Kirschenbaum,  Combat 
Control 

Consultants 
Walter  Clearwaters 
Prof.  Arthur  Westneat 
Gerson  Betancourt,  URI 


FAST 


i  GROUP 


Human  Brains 


•  Process  acoustic  signals  with  reasoning  and  learning 

•  Are  systems  with  (sensory)  inputs  and  (motor)  outputs 

•  Are  complex  systems  that  are  nonlinear  and  time-varying 

•  Have  short-term  and  long-term  memory  organized  with  schema 
based  world  model 

•  Have  slowly-changing  architectures  (synaptic  plasticity) 

•  Have  automatic  (subconscious)  and  controlled  (conscious) 

•  Can  nonlineariy  redirect  attention  in  response  to  stimulation 

•  Have  a  massively  parallel  architecture  with  extensive  feedback 

•  Contain  10-100  billion  neurons 

-*■  with  1,000  - 10,000  connections  to  other  neurons 
-►  with  nonlinear  and  time-varying 
with  sub-neuron  microtubule  structure 

•  Show  evidence  of  resonating  at  40  Hz 

•  Form  the  biological  inspiration  for  useful  computational  models 
Ilf? GROUP 
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Cognitive  Neuroscience  Network  Models 


•  Dynamically  Stable  Associative  Learning  (DYSTAL),  T.  Vogl  [1995],  GMU 

•  WRENN  constructed  neural  net  for  ATR,  R.  Woodall  [1994],  NUWC 

•  Sub-Neuron  Microtubules  increase  processing  power,  S.  Hameroff,  UAz 

•  ASCENT  human  vision  model,  for  NUWC  BTR  data,  L.  Finkel  [1994],  UPenn 

•  Synaptic  LTP  exceeded  conventional  sonar,  R.  Granger  [1994],  UCI 

•  Adaptive  Resonance  Theory  (ART),  S.  Grossberg,  G.  Carpenter  [1995],  BU 

•  Hardware,  A.  Maren  [1995],  Accurate  Automation  Corporation  (AAC) 

.  Streit’s  Probabilistic  Neural  Network  (SPNN),  R.  Streit  [1991],  NUWC 

•  Automatic  Target  Recognition  Neural  Net  H/W,  L.  Cooper,  Brown  U 

•  Neurally  Inspired  Contact  Estimation  (NICE),  C.  DeAngelis  [1995],  NUWC 

•  Auditory  Scene  Analysis  (ASA),  A.  Bregman 

•  Data-Driven  Computational  ASA  (CASA),  Cooke,  Brown,  Klassner 

•  Prediction-Driven  CASA,  Ellis 

•  Top-Down  Hybrid  Convergent  Model,  W.  Clearwaters,  A.  Westneat  etal. 

©lHiifep - 


Apply  CNS  to  Meet  the 
Underwater  Acoustic  Technical  Challenge 


Traditional  sonars: 

•  Rely  heavily  on  human  sight  and  hearing 

•  Overload  and  tire  human  operators 

•  Are  not  capable  of  handling  more  acoustic  inputs 

•  Leave  information  extraction  to  human  operators 

•  Have  labor  intensive  displays  (e.g.,  BTR) 

•  Require  manual  (human)  classification 


Today’s  sonars  met  yesterday’s  needs, 
but  will  not  meet  tomorrow’s  needs 


FAST 

group 
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Initital  Objectives 


•  Extend  CNS  and  Computational  Hearing  to 
underwater  acoustics  /  sonar 

•  Develop  a  specification  and  evaluation 
criteria  for  FAST  CNS  sonar 

•  Design,  build,  and  test  a  revolutionary 
computational  sonar  that  emulates  CNS 
human-brain  functions 


FAST 

GROUP 


Our  Vision: 


A  Revolutionary  CNS  Sonar  that  will: 


•  Replace  human  sonar  operators  with  fully 
automated  “silicon-based”  assistants  that 
recommend  timely  decisions  with  expert  or 
“ace”  abilities  to  a  human  machine  supervisor 

•  Perform  well  in  new  acoustic  environments 

•  Handle  an  order  of  magnitude  more  acoustics 

•  Fuse,  compress,  &  merge  data  into  information 

•  Display  needed  information  in  the  right  format, 
to  the  right  decision  maker,  at  the  right  time 

•  Adapt  to  new  tasks  by  learning  &  reasoning 
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I/O  Overview  for  CNS  Sonar 


Input  data  sources 


Output  (information) 


FAST 


GROUP 


Prediction-Driven  Element 
of  a  CNS  Sonar  Architecture 


Output  \ 
Information  J 


. .  I"  :  V/Predicted  >  - - - 

1  A  Features 

1 

• 

f  Acoustic  ^ 
l  Input  Data  J 

‘  C  Predict  'V- _ 

1 

1 

1 

i  V_„  ~  Schema  ; 

Modified  from  Ellis,  1996  MIT  Ph.D.  Thesis 


FAST 

GROUP 


Note  use  of  feedback,  memory, 
alertment,  and  world  model 
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CASA: 

Developing  a  CNS  Architecture 


CNS  ASA  CASA 


Technical  Status 


•  Identified  need  and  FULL  AUTOMATION  GRAND  CHALLENGE 

•  Published  JUA  article  on  CNS  Implications  to  sonar 

•  Completed  study  of  Computational  Acoustic  Scene  Analysis 
(CASA);  Arthur  Westneat,  1997 

•  Analyzed  Prediction-Driven  CASA  (Ellis,  1996  MIT  Ph.D.  Thesis) 
that  extended  Cooke’s  Data-Driven  CASA  approach 

•  Extended  sub-neuron  work  of  Prof.  S.  Hameroff  to  more 
completely  explain  brain  functions 

•  Developed  basis  for  showing  Artificial  Neural  Networks  (ANNs) 
achieve  Maximum  Likelihood  (optimum)  performance  (R.  Streit) 

•  Applied  Prof.  L.  Finkel  ASCENT  algorithm  to  sonar  BTR  data 

•  Met  with  small  business  building  ANNs  (L.  Cooper,  Nobel 
Laureate  and  Dr.  A.  Maren,  Accurate  Automation  Corporation) 

•  Held  workshop  on  CNS  Sonar 


Preliminary  Results  of  Applying  ASCENT 
Algorithm  to  Bearing-Time  Recorder  (BTR)  Data 


Time 

* 


SONAR  image 

ASCENT  Output 


Results  with  Dr. 
Finkel,  UPENN 
demonstrate  full 
automation  of 
visual  display  of 
acoustic  data 


Bearing 


•  Modeled  after  the  human  visual  processing  system 


•  ASCENT  extracts  salient  contours  from  a  real  BTR  display 


£flST_ 
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WRENN  Neural  Network 
Architecture 


ACOUSTIC 

INPUT 


(  NEURAL  T 
SENSOR 

'  NEURAL  DIRECTOR  ARRAY 

1 

PKOM  ARRAY  1 

^  «  -J 

1 

MEMORY  VECTOR 

v  .  ....  J 

1 

{  senstmt? 

*  CONTROL 

COMMON  VECTOR  ARRAY 

_  _  w 

1 

RECOGNITION  VECTORS 

^  -  ' . ■ . ^ 

CLASS 

1 

PKOM 

CLASSIFIER 

OUTPUT 


PKOM  =  Position  King  of  the  Mountain  (Winner  Take  All) 

©m&S,  FAST _ 

omJi  group 
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Results  of  Successfully  Applying 
WRENN  Algorithm  to  Underwater 
Porpoise  Data 


INPUT  (12  seconds) 


APPROX,  -3  dB  SNR  APPROX,  2  dB  SNR 


CLASS  1  OUTPUT  (Ascending  Tone  Class) 

pvv  r\ 


CLASS  2  OUTPUT  (Descending  Tone  Class) 

_ .. ,  —n - __ - ul 


\ 

APPROX.  14  dB  SNR 


J**\ 
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Open  Technical  Questions 

•  How  do  we  test  &  evaluate 

-+a  complex  time-varying,  nonlinear  system? 

-►learning,  reasoning,  and  adaptability? 

•  What  extensions  are  required  to  CNS  to  build  a  CNS  sonar? 

•  How  would  we  demonstrate  a  FASTWAR  CNS  sonar? 

•  How  does  the  internal  architecture  change  with  time? 

•  Is  problem  scaleable?  Is  a  CNS  sonar  demonstrable  in  small 
system?  Or  does  it  take  a  large  system?  How  many  neurons 
should  be  in  the  first  phase  test  bed  sonar  system? 

•  What  are  appropriate  architectures  for  our  CNS  sonar? 

•  How  will  we  “program”  our  CNS  sonar? 

•  How  will  our  CNS  sonar  learn? 

•  How  and  at  what  data  rates  will  we  stimulate  sonar? 

•  How  does  our  CNS  sonar  implement  the  subconscious  (automatic) 
and  conscious  (controlled)  mind? 

®ti©  wg- _ 
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Implications  of  Moore’s  Law 


Researchers  16  RACKS 


Conceptual  Program  (Timeline) 


Example 

Science  team 


Commerical  Sonars,  e.g.,  to 
_ monitor  fishing  treaty  compliance 

0  3  6  9 

TIME  (YEARS) 
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Overview  of  CNS  Thrusts 


TOPIC 

NUWC  Lead 

Partner 

CNS  Systems 

Clearwaters 

UNH  (A.  Westneat)  /  NUWC  (Keil) 

ASCENT 

Carter 

UPENN  (L.  Finkel)/  ONR  (Hawkins/Davis) 

JONESY 

Masakowski 

T.B.D. /CNO  (Morgan) 

START 

Shores 

BU  (G.  Carpenter)  /  ONR  (Hawkins/Harned) 

CASA 

Meier 

VU  (F.  Klassner)/OSU  (D.Wang) 

NOSIP 

Quazi/Katz 

MIT  (C.  Poon)/  ONR  (Davis) 

TSA 

Kirsteins 

UR1  (R.  Kumerasean)/ONR  (Gisiner) 

AAC/CMA 

Streit 

T.B.D. /DARPA  (Munoz) 

GUS 

Masakowski 

GaTech  (T.  Doll)  /  CNO  (Hanlon) 

World  Model 

Aaron 

T.B.D. 

FAST 

GROUP 


rltUt  WH. 


Project: 

CNS  Systems 

NUWC  leader(s): 

W.  Clearwaters  &  CNS  FAST 
Group 

Team  Members: 

A.  Westneat,  UNH,  various 
universities,  and  industrial 
partners 


Objective: 

Develop  several  CNS 
systems  based  upon 
human  brain  architecture 

Payoff: 

Revolutionary  underwater 
systems  that  cognitively 
and  automatically  handle 
ten  times  the  volume  of 
data 


Technical  Issues:  Milestones: 

Can  we  demonstrate  automatic  T.B.D. 

and  cognitive  machine  processing 
as  a  first  step  towards  Fully 
Automated  System  Technology 
(FAST)? 


FAST 

GROUP 
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Project: 

GTV  User  System  (GUS) 

NUWC  leader(s): 

Dr.  Yvonne  R.  Masakowski 

Team  Members: 

Dr.  Theodore  J.  Doll,  GA.  Tech 
Dr.  G.  Clifford  Carter,  NUWC 

Technical  Issues: 

Development  of  intelligent 
expert  systems  based  on 
perceptual  expertise  of 
operators 

Approach: 

Investigate  GaTech  Vision  (GTV) 
simulation  as  an  automatic 
recognition  system 


Objective: 

Enhance  sonar  displays, 
and  operator  training  and 
performance 

Payoff: 

Improved  recognition  & 
classification  of  sonar 
imagery  in  cluttered 
environment  in  decreased 
amount  of  time 

Milestones: 

•  Comparative  report  on 
developed  displays 

•  Briefing  and  video  tape 
demo  of  GUS 

•  Transition  to  sonar  system 


Project: 

ASCENT 

NUWC  leader(s): 

Dr.  Cliff  Carter,  NUWC 

Team  Members: 

Dr.  Lew  Meier,  NUWC 
Dr.  Leif  Finkel,  UPENN 

Technical  Issues: 

•  Implications  of  display 
thresholding  on  performance 

•  Output  interface  for  display  tracks 


Objective: 

Extend  human  eye  research 
to  automatically  recognize 
sonar  displays  with  human 
ability 

Payoff: 

Reduce  the  number  of  sonar 
operators  by  factor  of  3  while 
handling  10  times  the  data 

Milestones: 

•  Test  and  Evaluation 
Algorithms 

•  Refine  Algorithms,  Evaluate 
Performance 

•  Transition  to  sonar  product 
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Project: 

Tonotopic  Signal  Analysis  (TSA) 

NUWC  leaders ); 

Dr.  Ivars  Kirsteins 

Team  Members: 

Prof.  R.  Kumaresan,  URI 
Dr.  J.  Fay,  NUWC 
Dr.  S.  Mehta,  NUWC 

Technical  Issues: 

•  Development  of  mathematical 
algorithms 

•  Representation  of  radiated 
signatures 

•  Robustness  to  noise 


Objective: 

Human  listener-like  capability 
for  characterization  and 
classification  of  radiated 
signatures 

Payoff: 

Significantly  improved 
passive  sonar  multi-sensor 
detection,  classification, 
localization,  and  data  fusion 

Milestones: 

•  Development  of  mathematical 
steps  and  application  to  sea 
state 

*  Demonstration  using  sea  data 


•  Classification  performance  gains  •  Transition  to  sonar  product 
- 


Project: 

JONESY 

NUWC  leader: 

Dr.  Yvonne  R.  Masakowski 

Team  Members: 

Dr.  Susan  S.  Kirschenbaum,  NUWC 
Mr.  John  Cooke,  NUWC 

Technical  Issues: 

Development  of  intelligent  entropic 
system  based  on  extrapolating 
perceptual  strategies  of  expert 
operators 


FACT 


Objective: 

Enhance  displays  &  training 
based  on  perceptual  expertise 
of  sonar  operators 

Payoff: 

Reduced  training  time 
Decrease  in  decision  time 
Enhanced  human 
performance 

Milestones: 

•  Develop  displays  with 
strategies  or  perceptual  of 
expert 

•  Test  effectiveness  of  new 
training  system 

•  Implement  advanced  training 
system 
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Project: 

World  Model 

NUWC  leader(s): 

Dr.  Arnold  Aaron 

Team  Members: 

Dr.  John  Salisbury,  A&T 

Technical  Issues: 

Database  Architecture  for 
Cognitive  Systems 


Objective: 

Put  knowledge  into 
accessible  form 

Payoff: 

Rapid,  automated 
environmental  corrections 
Multi-contact  resolution 
Reduced  time 

Milestones: 

T.B.D. 


FAST 

GROUP 


Project: 

Nonlinear  Signal  Information  Processing 
(NOSIP)  for  CNS  Sonar 

NUWC  leader(s): 

Dr.  Azizul  H.  Quazi 
Dr.  Richard  Katz 

Team  Members: 

Dr.  C.  Poon,  MIT 
Dr.  C.  Baker,  UNC 
Dr.  J.  Salisbury,  A&T 

Technical  Issues: 

•  Estimation  of  Joint  Density 
function 

•  Robustness  of  nonlinearly 
invariant  measures 

Approach: 

•  Nonlinear  correlation  based  on 
information  theory 


Objective: 

Extend  nonlinear  signal 
information  for  enhanced 
acoustic  detection, 
classification,  and  localization 

Payoff: 

Improved  acoustic 
classification  and  localization 
in  USW 

Milestones: 

•  Collect  sea  data  and  generate 
simulated  signals;  analyze 
data 

•  Develop  algorithms 

•  Transition  to  sonar  system 


Project: 

CASA  in  Undersea  Environments 

NUWC  ieader(s): 

Dr.  Lewis  Meier 

Team  Members: 

Mr.  Art  Westneat,  UNH 
Dr.  Dan  Ellis,  UC-Berkeley 
Dr.  De  Liang  Wang,  Ohio  State 
Dr.  Frank  Klassner,  Villanova 

Technical  Issues: 

•  What  CASA  based  algorithms  and 
heuristic  procedures  are  useful  in 
undersea  systems? 

•  What  hardware  and  software 
architecture  are  best  to  implement 
these  algorithms? 


Objective: 

Augment  existing  systems  so 
human  operator  can  perform: 

•  Existing  tasks  better 

•  New  tasks 

Payoff: 

•  Provide  the  operator  with 
information  displays  of  the  acoustic 
scene 

•  Extend  the  systems  ability  to 
classify  a  wider  range  of  targets 

Milestones: 

•  Implement  existing  CASA  systems 
at  NUWC 

•  Test  with  simulated  and  real 
underwater  signals 

•  Modify  CASA  systems 

•  Demonstrate  CASA  at  sea 


Project: 

Early  Cognition  Model  for 
ACC / CMA 

NUWC  leader(s): 

Dr.  Roy  L.  Streit 

Team  Members: 

T.B.D. 

Technical  Issue: 

Validate  the  analogy  that 
multi-sensory  integration  is 
to  passive  ACC  as  early 
cognition  is  to  CMA 


Objective: 

Demonstrate  joint  Acoustic  Contact 
Correlation  (ACC)  and  Contact 
Motion  Analysis  (CMA)  processing 
based  on  mulit-sensory  integration 
and  early  cognition 

Payoff: 

Superior  automated  joint  ACC/CMA 
performance  when  compared  to 
current  non-integrated  hierarchical 
approach  of  first  ACC,  then  CMA 

Milestones: 

•  Develop  CNS  Algorithms 

•  Test,  refine,  demonstrate 

•  Transition  to  advanced  demo 


FAST 

GROUP 
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Project: 

START 

NUWC  leader(s): 

Dr.  James  P.  Shores  (NUWC) 

Team  Members: 

Mr.  Gerson  Betancourt,  URI 
Dr.  G.  Carpenter,  BU 

Technical  Issues: 

•  Will  ART  be  applicable  to 
underwater  acoustic  signals? 

•  What  gain  is  provided? 


Objective: 

Extend  Adaptive  Resonance 
Theory  (ART)  to  demonstrate 
Sonar  Transient  Adaptive 
Resonance  Theory  (START) 
classification 

Payoff: 

Improved  accuracy  and  speed  of 
classification  with  full  automation 

Milestones: 

•  Install  in  sonar  lab 

•  Test  with  sea  data 

•  Transition  to  sonar 


Summary 


•  Problem 

-►  Unknown  /  unpredictable  acoustic  tasks 
-►Variable  &  complex  ocean  acoustic  environments 
-►Today  sonar  operators  are  overloaded  with  data 
-►  Future  sensors  and  communications  with  off- 
board  sensors  will  further  increase  acoustic  load 

•  Solution 

-►  Exploit  and  extend  emerging  knowledge  of  brain 
and  Cognitive  Neuroscience  (CNS) 
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Conclusions 


•  Making  progress  understanding  and  extending 
Cognitive  Neuroscience  (CNS)  to  Sonar 

•  Offering  Researchers  an  opportunity  to  form 
cooperative  partnerships  with  NUWC  to 
advance  Fully  Automated  System  Technology 
(FAST)  Cognitive  Neuroscience  (CNS)  Sonar 
Research  &  Development 
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Traditional  sonars: 

•  Rely  heavily  on  human  sight  and  hearing 

•  Overload  and  tire  human  operators 

•  Are  not  capable  of  handling  more  acoustic  inputs 

•  Leave  information  extraction  to  human  operators 

•  Have  labor  intensive  displays  (e.g.  BTR) 

•  Require  manual  (human)  classification 

Fully  Automated  System  Technology  (FAST)  CNS  Sonars 
are  described  at  the  poster  session 
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